Measuring vehicle speeds with an uncalibrated camera

ABSTRACT

Measuring speed of a vehicle in a road environment. During calibration, multiple images are captured of a calibration vehicle traveling at a known ground speed. A calibration image feature is located in the image of the calibration vehicle. An optical flow of the calibration image feature is computed to determine a model between an image speed of the calibration image feature and the known ground speed of the calibration vehicle. During speed measurement, multiple images are captured of a target vehicle traveling along a road surface at unknown ground speed. A target image feature may be located in an image of the target vehicle. An image speed may be computed of the target image feature. The model may be applied to determine the ground speed of the target vehicle from the image speed of the target image feature.

CROSS REFERENCE TO RELATED APPLICATION

The present application claims priority to Great Britain PatentApplication No. 2015434.0, filed Sep. 30, 2020, the contents of whichare incorporated herein in its entirety.

BACKGROUND Technical Field

The present invention relates to usage of traffic cameras formeasurement of vehicle speed.

Description of Related Art

Modern speed enforcement involves use of a speed measuring device formeasuring vehicle speeds and triggering a camera to capture an image toidentify the speeding vehicle and driver to whom a citation may beissued. An accurate vehicle speed measurement is necessary for lawenforcement of speeding. A number of methods exist for measuring vehiclespeeds. RADAR, “radio detection and ranging”, is a technology that useselectromagnetic radio waves to detect moving vehicles and measure theirspeeds. LiDAR uses pulsed laser light instead of radio waves to measurevehicle speeds. Digital image processing has been suggested formeasuring speed of moving vehicles in a road environment.

BRIEF SUMMARY

Various methods and systems are disclosed herein for measuring speed ofa vehicle in a road environment. During calibration, multiple images arecaptured of a calibration vehicle traveling at a known ground speed. Acalibration image feature is located in the image of the calibrationvehicle. An optical flow of the calibration image feature is computed todetermine a model between an image speed of the calibration imagefeature and the known ground speed of the calibration vehicle. Duringspeed measurement, multiple images are captured of a target vehicletraveling along a road surface at unknown ground speed. A target imagefeature may be located in an image of the target vehicle. An image speedmay be computed of the target image feature. The model may be applied todetermine the ground speed of the target vehicle from the image speed ofthe target image feature. Image speed relative to ground speed may beproportional to a distance from a vanishing point of an image featureand proportional to a vertical image coordinate measured from avanishing point. Image speed relative to ground speed of the vehicle maybe proportional to a horizon slope factor related to camera roll aboutthe horizon and a horizontal image coordinate measured from thevanishing point. The model may be a parameterized model includingmultiple parameters. The model may be determined by determining theparameters of the model. The parameters of the model may include: aproportionality constant C relating ground speed V to image speed v,image coordinates x0 and y0 of a vanishing point in an image plane, andthe horizon slope factor related to camera roll about the horizon.

Various non-transitory computer-readable storage media storeinstructions that, when executed by a processor, cause the processor toperform a method as disclosed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is herein described, by way of example only, withreference to the accompanying drawings, wherein:

FIG. 1 illustrates a camera or pinhole coordinate system which relatesworld-space coordinates (X,Y,Z) to image coordinates (x,y);

FIG. 2 illustrates schematically a top view of a system including acamera viewing a road environment, according to embodiments of thepresent invention;

FIG. 2A illustrates an image viewed by the camera in FIG. 2 ;

FIG. 3 is a simplified flow diagram of a calibration stage of a method,according to embodiments of the present invention; and

FIG. 4 is a simplified flow diagram of speed measurement method oftarget vehicles, according to embodiments of the present invention.

The foregoing and/or other aspects will become apparent from thefollowing detailed description when considered in conjunction with theaccompanying drawing figures.

DETAILED DESCRIPTION

Reference will now be made in detail to features of the presentinvention, examples of which are illustrated in the accompanyingdrawings, wherein like reference numerals refer to like elementsthroughout. The features are described below to explain the presentinvention by referring to the figures.

By way of introduction, aspects of the present invention are directed toa system and method for accurately measuring the speeds of vehiclesalong a stretch of road. A planar road model may be assumed. The presentmethod as disclosed herein may be used to retrofit, with only a softwareupgrade, an existing camera already in use for license platerecognition, by way of example. Speed measurements, according toembodiments of the present invention may be used to monitor localtraffic and/or trigger an issuance of a citation for a speedingviolation.

Once setup, the system may be used to measure respective speeds ofmultiple vehicles within the camera's field of view (FOV),simultaneously including vehicles travelling in opposite directions, toand from the camera. The system and method as disclosed herein includestwo stages of operation.

(1) An initial calibration stage during which images are captured, i.e.,video, of a sample of vehicles travelling at known speeds.

(2) a measurement stage where the system may operate continuously,measuring and logging vehicle speeds in real-time.

Referring now to the drawings, reference is now made to FIG. 1 whichillustrates camera or pinhole projection which relates a point P(X,Y,Z)in world space Cartesian coordinates to a point p(x,y) image coordinateson image plane 8 where X is the horizontal Cartesian coordinate in worldspace, Y is the vertical Cartesian coordinate in world space and Z isthe direction along the optical axis of the camera. The origin O ofcamera projection is at the pinhole, image plane 8 is in reality behindthe origin at focal length f with the image inverted Image plane 8 isshown in the projection of FIG. 1 in a symmetric position with anon-inverted image in front of origin O at a distance focal length f sothe centre of the image plane 8 is at world space coordinates (0,0,f).The following equations approximate the relation between imagecoordinates x,y and world space coordinates X,Y,Z assuming camera orpinhole projection:

${x(t)} = {{f\frac{X(t)}{Z(t)}{y(t)}} = {f\frac{Y(t)}{Z(t)}}}$

Reference is now also made to FIG. 2 , which illustrates schematically asystem 20, according to embodiments of the present invention, a viewfrom above of a road environment. System 20 illustrates fixed in a roadenvironment, a camera 2 with field of view (FOV) with the optical axis Zof camera 2 parallel to the direction of motion of vehicles 18. Usingthe camera coordinate system as shown in FIG. 1 , the motion of avehicle 18 is modelled as starting from (X0, Y0, 0), a point in a planeperpendicular to the camera optical axis, and moving along a straighttrajectory in three dimensions, toward a point (X(t), Y(t), Z(t)). Thevehicle position is imaged on image plane 8 as being at (x(t), y(t)). Aprocessor 4 is shown connected to camera 2 configured to capture imageframes from the road environment.

Reference is now also made to FIG. 2A which schematically illustratesimage plane 8 imaging the road environment as shown in FIG. 2 .Vanishing point (x0, y0) is shown, the point on image plane 8 to whichthe trajectories of vehicle 18 converge as Z(t) approaches infinity.Arrows emanating from images of vehicles 18 moving along the road in thepositive Z direction indicate respective image velocity vectors in thedirection of the vanishing point. Length of the arrows represents speedv in image space.

Defining image coordinates relative to the vanishing point: Δx=x−x0 andΔy=y−y0, a model relating speed v measured in image space on image plane8 and ground speed V of the vehicle is a ratio:

$\begin{matrix}{\frac{v}{V} = {{C( {{a\Delta x} - {\Delta y}} )}\sqrt{( {( {\Delta x} )^{2} + ( {\Delta y} )^{2}} )}}} & (1)\end{matrix}$

where C is a proportionality constant and a is a horizon slope factorrelated to roll angle of the camera about the optical axis. If thecamera horizontal x axis is exactly parallel with the horizon, then a=0.Using equation (1), parameters: C, a, x0 and y0 may be determined duringthe calibration phase when vehicle speed is known. Subsequently, duringthe measurement stage, equation (1) may be used to determine respectiveground speeds V from speed v measurements in image plane 8 of imagefeatures of vehicles.

Still referring to FIG. 2A, a bounding box 7 is shown with a broken linetightly fitting an image of a vehicle 18. When using bounding box 7, x′may be defined as the mid-point of bounding box 7 along the horizontalimage axis x; and measured relative to vanishing point horizontalcoordinate x0\ is:Δx′=x′−x0

Similarly, y′ may be defined along the vertical image axis y as thebottom of bounding box 7; and relative to the vertical coordinate y0 ofthe vanishing point is:Δy′=y′−y0

When using bounding box 7, we can improve upon equation (1) by assumingthat the bottom of bounding box 7 is on the road to yield:

$\begin{matrix}{\frac{v}{V} = {{C( {{a\Delta x^{\prime}} - {\Delta y^{\prime}}} )}\sqrt{( {( {\Delta x} )^{2} + ( {\Delta y} )^{2}} )}}} & (2)\end{matrix}$Calibration Stage:

Calibration stage is for initial setup which, in some embodiments of thepresent invention, may include operator supervision. As long as camera 2remains fixed, the initial setup may be performed only once. There aretwo purposes to calibration mode: (i) to map regions of the road thatare straight and planar without dips or turns, and (ii) to find themodel parameters, C, a, x0 and y0.

Reference is now made to FIG. 3 , a flow diagram of a calibration stage30 according to embodiments of the present invention. Multiple imageframes are captured (step 31) of vehicles travelling along a road withknown respective speeds Image features in images of the travellingvehicles are located, (step 33) Image features are tracked between imageframes. ORB may be used to match features and track between imageframes. (Ethan Rublee, Vincent Rabaud, Kurt Konolige, Gary R. Bradski:ORB: An efficient alternative to SIFT or SURF. ICCV 2011: 2564-2571).

Optical flow of the image features may be computed between two or moreimage frames to produce a set of motion vectors. Vehicle images may bestabilised by considering optical flow outside the vehicle images, i.e.the part of the image that is not expected to be moving. Any other knownmethods for stabilization may be used. Spurious optical flow vectors maybe filtered out. Spurious optical flow vectors include: vectors that donot emanate or end in a bounding box. Vectors that are too short andrepresent noise. Motion vectors with directions very different from mostof the optical flow vectors or are too long and may represent an opticalflow feature mismatch are outliers and preferably filtered out.

Filtered optical flow vectors are input into a model based on equation(1) to optimally determine (step 35) parameters C, a, x0 and y0. Someregions in image plane 8 may consistently have improved fits to themodel, other regions may have worse fits to the model. Image plane 8 maybe mapped so that better fitting images are used for parameterdetermination. (step 35)

Speed Measurement Stage

Reference is now made to FIG. 4 which is a simplified flow diagram ofspeed measurement of target vehicles 18, according to embodiments of thepresent invention. Multiple image frames are captured (step 41) ofvehicles travelling along a road with unknown respective speeds Imagefeatures in images of the travelling vehicles are located, (step 43).

As in the calibration stage, similar processing steps may be performed.Target image features are tracked between image frames. Optical flow ofthe image features may be computed between two or more image frames toproduce a set of motion vectors. Vehicle images may be stabilised andspurious optical flow vectors may be filtered out. Filtered optical flowvectors are input into the model based on equation (1) including modelparameters C, a, x0 and y0 to optimally determine (step 45) the groundspeed V of the target vehicle from the image speeds of the target imagefeatures.

The term “image feature” as used herein is one or more portion of animage such as a corner or “blob” and may be detected by any method inthe art of digital image processing.

The term “optic flow” or “optical flow” as used herein refers to patternof apparent motion of image features in image space caused by motion ofan object relative to the camera.

The term “image speed” as used herein is the magnitude of an opticalflow vector in an image plane divided by time.

The term “calibration” as used herein is not “camera calibration” whichgenerally refers to determining internal camera parameters: e.g., focallength, distortion, and external camera parameters: position coordinatesand/or angular coordinates of the camera relative to the road. Thesecamera coordinates are not generally required to be known a priori inthe present method. The term “calibration” as used herein refers to theinitial stage of the method disclosed herein which uses images of avehicle travelling at known speed to determine parameters of a modelrelating ground speed to image speed.

The term “vanishing point” as used herein is a point (x0,y0) in imagespace to which an image of an object emanates as the object approachesinfinite distance from the camera along the positive optical axis.

The transitional term “comprising” as used herein is synonymous with“including,” and is inclusive or open-ended and does not excludeadditional, unrecited elements or method steps. The articles “a” and“an” are used herein, such as “a vehicle,” “an image feature,” have themeaning of “one or more,” that is “one or more vehicles,” “one or moreimage features.”

The various aspects, embodiments, implementations or features of thedescribed embodiments can be used separately or in any combination.Various aspects of the described embodiments can be implemented bysoftware, hardware or a combination of hardware and software. Thedescribed embodiments can also be embodied as computer readable code ona non-transitory computer readable medium. The non-transitory computerreadable medium is any data storage device that can store data which canthereafter be read by a computer system. Examples of the non-transitorycomputer readable medium include read-only memory, random-access memory,CD-ROMs, HDDs, DVDs, magnetic tape, and optical data storage devices.The non-transitory computer readable medium can also be distributed overnetwork-coupled computer systems so that the computer readable code isstored and executed in a distributed fashion.

All optional and preferred features and modifications of the describedembodiments and dependent claims are usable in all aspects of theinvention taught herein. Furthermore, the individual features of thedependent claims, as well as all optional and preferred features andmodifications of the described embodiments are combinable andinterchangeable with one another.

Although selected features of the present invention have been shown anddescribed, it is to be understood the present invention is not limitedto the described features.

The invention claimed is:
 1. A method comprising: during calibration,capturing a plurality of images of a calibration vehicle traveling at aknown ground speed; locating a calibration image feature in an image ofthe calibration vehicle; computing an optical flow of the calibrationimage feature thereby determining a model between an image speed of thecalibration image feature and the known ground speed of the calibrationvehicle; during speed measurement, capturing a plurality of images of atarget vehicle traveling along a road surface at unknown ground speed;locating a target image feature in an image of the target vehicle;computing an image speed of the target image feature; and applying themodel to determine the ground speed of the target vehicle from the imagespeed of the target image feature.
 2. The method of claim 1, whereinimage speed relative to ground speed is proportional to a distance froma vanishing point of an image feature and proportional to a verticalimage coordinate measured from a vanishing point.
 3. The method of claim1, wherein image speed relative to ground speed of the vehicle isproportional to a horizon slope factor related to camera roll about thehorizon and a horizontal image coordinate measured from the vanishingpoint.
 4. The method of claim 1, wherein the model is a parameterizedmodel including a plurality of parameters, wherein said determining themodel includes determining the parameters of the model.
 5. The method ofclaim 4, wherein the parameters of the model include: a proportionalityconstant C relating ground speed V to image speed v, image coordinatesx₀ and y₀ of a vanishing point in an image plane and a horizon slopefactor related to camera roll about the horizon.
 6. A system including aprocessor and a camera, the system configured to: during calibration,capture a plurality of images of a calibration vehicle traveling at aknown ground speed; locate a calibration image feature in an image ofthe calibration vehicle; compute an optical flow of the calibrationimage feature thereby determine a model between an image speed of thecalibration image feature and the known ground speed of the calibrationvehicle; during speed measurement, capture a plurality of images of atarget vehicle traveling along a road surface at unknown ground speed;locate a target image feature in an image of the target vehicle; computean image speed of the target image feature; and apply the model todetermine the ground speed of the target vehicle from the image speed ofthe target image feature.
 7. The system of claim 6, wherein image speedrelative to ground speed is proportional to a distance from a vanishingpoint of an image feature and proportional to a vertical imagecoordinate measured from a vanishing point.
 8. The system of claim 6,wherein image speed relative to ground speed of the vehicle isproportional to a horizon slope factor related to camera roll about thehorizon and a horizontal image coordinate measured from the vanishingpoint.
 9. The system of claim 6, wherein the model is a parameterizedmodel including a plurality of parameters, wherein said determining themodel includes determining the parameters of the model.
 10. The systemof claim 6, wherein the parameters of the model include: aproportionality constant C relating ground speed V to image speed v,image coordinates x₀ and y₀ of a vanishing point in an image plane and ahorizon slope factor related to camera roll about the horizon.
 11. Anon-transitory computer-readable storage medium storing instructionsthat, when executed by a processor, cause the processor to perform amethod comprising: during calibration, capture a plurality of images ofa calibration vehicle traveling at a known ground speed; locate acalibration image feature in an image of the calibration vehicle;compute an optical flow of the calibration image feature therebydetermine a model between an image speed of the calibration imagefeature and the known ground speed of the calibration vehicle; duringspeed measurement, capture a plurality of images of a target vehicletraveling along a road surface at unknown ground speed; locate a targetimage feature in an image of the target vehicle; compute an image speedof the target image feature; and apply the model to determine the groundspeed of the target vehicle from the image speed of the target imagefeature.
 12. The non-transitory computer-readable storage medium ofclaim 11, wherein image speed relative to ground speed is proportionalto a distance from a vanishing point of an image feature andproportional to a vertical image coordinate measured from a vanishingpoint.
 13. The non-transitory computer-readable storage medium of claim11, wherein image speed relative to ground speed of the vehicle isproportional to a horizon slope factor related to camera roll about thehorizon and a horizontal image coordinate measured from the vanishingpoint.
 14. The non-transitory computer-readable storage medium of claim11, wherein the model is a parameterized model including a plurality ofparameters, wherein said determining the model includes determining theparameters of the model.
 15. The method of claim 14, wherein theparameters of the model include: a proportionality constant C relatingground speed V to image speed v, image coordinates x₀ and y₀ of avanishing point in an image plane and a horizon slope factor related tocamera roll about the horizon.